Skip to content

replace requests with urllib #4973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 23, 2021
Merged

replace requests with urllib #4973

merged 2 commits into from
Nov 23, 2021

Conversation

pmeier
Copy link
Collaborator

@pmeier pmeier commented Nov 22, 2021 •
edited
Loading

Closes #4941. With this we lose the ability to send the confirm token, since urllib has no support for it. I've checked all of our GDrive resources in the datasets and none required it. TBH, I don't think we ever had a case where it was needed.

In case the confirm token is indeed needed in the future, I suggest we hard depend on requests, since urllib is only meant for very simple uses cases.

cc @pmeier

@pmeier pmeier requested review from malfet and datumbox November 22, 2021 16:00
@facebook-github-bot
Copy link

facebook-github-bot commented Nov 22, 2021 •
edited
Loading

💊 CI failures summary and remediations

As of commit 8ae1c48 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

NicolasHug
NicolasHug approved these changes Nov 22, 2021
View reviewed changes
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pmeier , LGTM when green

@pmeier pmeier merged commit 8d25de7 into pytorch:main Nov 23, 2021
@pmeier pmeier deleted the replace-requests branch November 23, 2021 07:52
@prabhat00155 prabhat00155 mentioned this pull request Nov 30, 2021
Closed
facebook-github-bot pushed a commit that referenced this pull request Nov 30, 2021
@prabhat00155 @facebook-github-bot
[fbsync] replace requests with urllib (#4973)
442de62 
Reviewed By: NicolasHug

Differential Revision: D32694312

fbshipit-source-id: b2cbebbf578e91a26b59491cbd3d4b5392152c0c
@pmeier
Copy link
Collaborator Author

pmeier commented Dec 7, 2021

While working on #5035, I realized we unfortunately sometimes need the confirm token. This happens if GDrive is unable to perform a virus scan and is asking the user for confirmation to proceed. This regularly happens for large files. Thus, I would follow my suggestion in the top comment to revert this PR and afterwards make requests a hard dependency.

@datumbox
Copy link
Contributor

datumbox commented Dec 7, 2021

@pmeier There were a couple of PRs that removed the functionality (including cleaning up a private method), so clicking the revert button here wont do the trick. You might need to do a bit more to undo the change. Concerning adding requests as a hard dependency, I'm OK with it.

pmeier added a commit to pmeier/vision that referenced this pull request Dec 7, 2021
@pmeier
Revert "replace requests with urllib (pytorch#4973)"
5676d1e 
This reverts commit 8d25de7.
pmeier added a commit to pmeier/vision that referenced this pull request Dec 7, 2021
@pmeier
Revert "replace requests with urllib (pytorch#4973)"
f02e721 
This reverts commit 8d25de7.
@pmeier pmeier mentioned this pull request Dec 7, 2021
Merged
pmeier added a commit that referenced this pull request Dec 8, 2021
@pmeier
Revert removing requests and make it a hard dependency instead (#5047)
4282c9f 
* Revert "[FBcode->GH] remove unused requests functionality (#5014)"

This reverts commit 33123be.

* Revert "replace requests with urllib (#4973)"

This reverts commit 8d25de7.

* add requests as hard dependency

* install library stubs in CI

* fix syntax

* add requests to conda dependencies

* fix mypy CI
facebook-github-bot pushed a commit that referenced this pull request Dec 9, 2021
@facebook-github-bot
[fbsync] Revert removing requests and make it a hard dependency ins…
a733b37 
…tead (#5047)

Summary:
* Revert "[FBcode->GH] remove unused requests functionality (#5014)"

This reverts commit 33123be.

* Revert "replace requests with urllib (#4973)"

This reverts commit 8d25de7.

* add requests as hard dependency

* install library stubs in CI

* fix syntax

* add requests to conda dependencies

* fix mypy CI

Reviewed By: NicolasHug

Differential Revision: D32950928

fbshipit-source-id: de0ed9a1c7cb5ca6e1b2a30db76e3ccc2008d25c
@datumbox datumbox added the revert(ed) For reverted PRs, and PRs that revert other PRs label Feb 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NicolasHug NicolasHug NicolasHug approved these changes

@malfet malfet Awaiting requested review from malfet

@datumbox datumbox Awaiting requested review from datumbox

Assignees
No one assigned
Labels
bc-breaking ciflow/default cla signed dependency issue module: datasets revert(ed) For reverted PRs, and PRs that revert other PRs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Replace requests by urllib
4 participants
@pmeier @facebook-github-bot @datumbox @NicolasHug